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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .1 7(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03 
August 2009 has been entered. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-14, 20, 21, and 23-26 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Masukura et al. (Pub. No.: US 2004/0148640) and further in view 
of Yamauchi et al. (US Pat. No.: 5,907,659) and Addington et al. (Pub. No.: US 
2004/0025181). 

As to claims 1,2, 10, and 24-26 Masukura teaches a system and associated 
method for processing a video program on a frame-by-frame basis in which the 
metadata changes temporally to reflect changes in the conversion information. 
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Masukura further teaches that the video program and metadata may be transmitted to a 
remotely located output device "by way of a network or broadcast waves" [0029]. 
Masukura et al. teaches a method for processing a video image in which the processing 
instructions conversion information are encoded in metadata associated with the 
content and that said metadata includes aspect ratio conversion information [0071]. As 
to the limitation that the metadata "dynamically changes," paragraphs 0053-54 of 
Masukura teach that "...the processing is basically carried out frame-by-frame... the 
metadata may be read as needed during processing. " Further, Fig. 3 and 5 teach that 
the metadata associated with each frame or temporal region may specify different 
conversion or processing information. 

Though paragraph 0071 of Masukura teaches that the "restriction information 
about the relative frame is acquired (step S71) from the metadata .. .The restriction 
information includes the number of pixels and the aspect ratio of an apparatus that uses 
the output moving pictures, " it does not explicitly teach converting from a first aspect 
ratio to a second aspect ratio. In an analogous art, Yamauchi teaches steps for 
converting a display from a first aspect ratio to a second aspect ratio. Fig. 1 A-2B of 
Yamauchi teach converting from a first aspect ratio to a second aspect ratio. Further, 
Fig. 13A-C and 27A-D of Yamuchi teach that the conversion information can change 
from frame-to-frame (e.g., to accommodate subtitle placement). As Yamauchi teaches 
methods for performing aspect ratio conversion, the combination of the conversion and 
processing information taught by Masukura, which allows different portions of a frame to 
be altered differently, with the aspect ratio conversion taught by Yamauchi would have 
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been obvious to one of ordinary skill in the art at the time of the invention since such 
combination would have yielded results that would have been predictable to one of 
ordinary skill in the art at the time of the invention. This would have been desirable as it 
would have enabled a wide range of video processing functionality to be available to 
broadcast programming which enhance the viewer's experience. 

As to the limitation that the video program be transmitted to a geographically 
different location, Masukura teaches, "...the output moving picture storage unit 106 may 
exist in a remote place by way of a network or broadcast waves" [0033]. One of 
ordinary skill in the art at the time of the invention would have recognized that as the 
conversion information of Masukura changes during the program, broadcasting the 
video program of Masukura would necessarily comprise broadcasting temporally or 
dynamically changing metadata in a manner cumulative with the method of claims 1 and 
2. 

As Yamauchi teaches converting frames of a video program from a first aspect 
ratio to a second aspect ratio on a frame-by-frame basis and Masukura teaches 
processing frames of a video program according to metadata which changes 
dynamically during said video program, one of ordinary skill in the art at the time of the 
invention would have recognized that the combined teachings of Masukura and 
Yamauchi, when considered as a whole, would provide dynamically changing metadata 
containing instructions facilitating the processing of frames of a video stream, said 
processing including aspect ratio conversion, as recited in the method of claims 1 and 2. 
However, the combination of Masukura and Yamauchi, as analyzed above does not 
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explicitly teach the newly-presented limitation of " integrating the video conversion 
information with the stream of images of the video program having the first aspect ratio 
to produce a combined set of video data, the combined set of video data formatted for 
viewing in both the first aspect ratio and in the second aspect ratio according to the 
conversion information ." In an analogous art, Fig. 2 of Addington teaches a video 
distribution system comprising Multiplexor and Scheduling Processor 280, which 
combines live broadcast content and metadata into a single transport stream that is 
broadcast to the viewer [0037]. As the processing and aspect ratio conversion 
information taught by Masukura and Yamauchi is equivalent to the metadata taught by 
Addington and further in light of Masukura's teaching that said metadata may be stored 
remotely from the user and transmitted through a network or broadcast waves, it would 
have been obvious to one of ordinary skill in the art at the time of the invention that the 
distribution system taught by Addington which multiplexes video information and 
metadata into a single transport stream could be utilized to distribute the video 
processing and aspect ratio conversion metadata taught by Masukura and Yamauchi. 
One of ordinary skill in the art would have recognized this as a combination of known 
elements in the art that would have yielded predictable results. One of ordinary skill in 
the art at the time of the invention would have recognized that this would have been 
advantageous as it allows for the video content and metadata to be broadcast on one 
channel thus increasing the efficiency of the system. 

Addington further teaches that the multiplexed transport stream is then delivered 
to a receiver 20 which forwards said transport stream containing both video content and 
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metadata to a set top box 270. The set top box 270 is equivalent to the recited 
" receiving display device at a second location ." 

As to the distribution system recited in claim 10, Masukura teaches that the both 
the input and output moving-picture and metadata storage units may exist "in a remote 
place by way of a network or broadcast waves" [0029, 0033]. As to the recited 
distribution point, Masukura teaches that "...a video camera or a broadcast wave tuner 
may be used as the input moving-picture storage unit" [0027] which is equivalent to the 
recited distribution point. Examiner maintains that broadcasting the video program of 
Masukura would inherently comprise temporally or dynamically changing metadata in a 
manner cumulative with the system of claim 10. 

As to the limitations recited in claims 24 and 25 regarding a first and second 
subset of the stream of images, the temporal regions taught by Masukura are equivalent 
to the recited subsets. Fig. 5 teaches that the metadata may contain different 
conversion or manipulation data for each temporal region. 

As to claim 3, Masukura teaches that the metadata can be adapted to adjust for 
the aspect ratios of various display devices [0071] which is consistent with the recited 
"plurality of recited target aspect ratios." 

As to claim 4, Fig. 1B-2B of Yamauchi teaches the recited shrinking. 

As to claims 5 and 23, paragraph 0071 of Masukura teaches: 

restriction information about the relative frame is acquired (step S71) from the 
metadata. The restriction information is information to limit the position of the cutout 
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region. The restriction information includes the number of pixels and the aspect ratio of 
an apparatus that uses the output moving pictures. 

Using the restriction information to limit the position of a cutout region is 
equivalent to the limitation recited in claim 5 that "the video conversion information 
specifies a portion of the video program to display as the second aspect ratio." 

As to claim 6, Fig. 4-8 of Masukura teach that different areas of the image can be 
scaled differently as recited; further, Masukura teaches that the height or width of the 
image may be scaled differently to accommodate the aspect ratio of the display device 
[0082]. 

As to claims 7, 8, and 21 , the recited "rotating or mirroring" and "plurality of 
discreet portions" is taught by Fig. 4 and 6 of Masukura as applied to claim 6. The 
shaded areas of Fig .4 and 6 are equivalent to the recited "plurality of discreet portions," 
and their manipulation (movement, rotation, etc.) evident in the difference between the 
two figures is equivalent to the recited "rotating or mirroring" and "different 
transformations for different portions." 

As to claim 9, the recited computer-readable medium and computer-executable 
instructions are inherent in a system capable of receiving and processing digital video, 
such as that taught by Masukura. 

As to claim 1 1 , Fig .1 of Masukura teaches an output video display coupled to a 
processed video generator. 

As to claims 12 and 13, paragraphs 0053-54 of Masukura (cited above) teach 
that the processing information is read from the metadata on a frame-by-frame basis, or 
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as needed by the video processor; it is inherent that this would allow the video 
processor to change conversion to a third aspect ratio mid-stream if such change were 
indicated by the metadata. 

As to claim 14, Fig. 4 and 6 of Masukura teach the recited "first and second 
cutout" and Fig. 1 teaches that the cutout-processing instructions are contained within 
the metadata. 

As to claim 20, Yamauchi teaches a method which includes converting a video 
image from a first aspect ratio to a second aspect ratio. In Fig. 1 B, an image originally 
in a 16:9 aspect ratio is converted into a 4:3 aspect ratio. The dashed lines overlaid on 
the 1 6:9 image in Fig. 1 would be the equivalent of aperture 1 01 2 shown in Fig. 1 0 of 
the instant application. Fig. 1 B of Yamauchi further teaches that only the portion of the 
16:9 image corresponding with the overlay is displayed in the converted 4:3 image. 
Conversely, Fig. 1C of Yamauchi teaches a method for converting from an original 4:3 
aspect ratio to a 16:9 aspect ratio. Further, the shaded areas of Fig .4 and 6 of 
Masukura would be equivalent to the recited apertures or cutouts and movement and 
teach the recited "manipulations," as analyzed w/r/t claims 7 and 8. 

Claim 22 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Masukura in view of Addington and Yamauchi as applied to claims 1 and 10 
above, and further in view of Duffield et al. (US Pat. No.: 5,461,427, previously 
cited). 

As analyzed above, Yamauchi in view of Masukura teaches the method of claim 
1 including the use of metadata to facilitate conversion from a first aspect ratio to a 
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second aspect ratio, but does not explicitly teach the use of one or more video streams 
of the program. Duffield teaches that the same program may be simultaneously 
broadcast (simulcast) in both NTSC and HDTV standards so to accommodate viewers 
with NTSC-compatible televisions as well as users with HDTV sets. Duffield teaches 
that such simulcasting is necessary to service users of both standards because, "the 
HDTV system uses a 16:9 aspect ratio while the NTSC system has a 4:3 aspect ratio, 
the HDTV system will have 1 125 television scan lines while the NTSC system has only 
525" (Col. 1 ). Duffield further teaches a receiver which can receive and process both 
NTSC and HDTV signals and, "includes a controller responsive to user input for 
associating a single label with a pair of television channels, one being an NTSC-signal 
carrying channel, and the other being an HDTV-signal carrying channel" (Col. 2). Figs. 
2a and 2b of Duffield show the same program displayed in both the 16:9 and 4:3 aspect 
ratios, respectively. As simulcasting the same video program in multiple formats to 
accommodate viewers with different display types was known in the art of video 
distribution, it would have been obvious to one of ordinary skill at the time of the 
invention to stream or broadcast the same program in multiple aspect ratios or display 
formats to allow users with different display capabilities to enjoy the same program. 
Since Duffield's disclosure was published in 1995, it does not explicitly refer to 
embodying video programs in a "content stream," but transmitting a video program in a 
content stream is an application of existing technologies that would have been obvious 
to one of ordinary skill in the art at the time of the invention. 
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Response to Arguments 

Applicant's arguments with respect to claims 1,10, and 24 have been considered 
but are moot in view of the new ground(s) of rejection. 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. Lude et al. (Pub. No.: 2002/0184642) discloses a similar system 
to that of Addington wherein supplemental data is combined with a video program into a 
single stream prior to being transmitted to the user. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan Stronczer whose telephone number is (571) 270- 
3756. The examiner can normally be reached on 7:30 AM - 5:00 PM (EDT), Monday- 
Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Brian T. Pendleton can be reached on (571) 272-7527. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Ryan Stronczer/ 
Examiner, Art Unit 2425 

/Brian T. Pendleton/ 

Supervisory Patent Examiner, Art Unit 2425 



